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Abstract. In the weakened 16th Hilbert's Problem one asks for a bound of 
the number of hmit cycles which appear after a polynomial perturbation of 
a planar polynomial Hamiltonian vector field. It is known that this number 
is finite for an individual vector field. In the multidimensional generalization 
of this problem one considers polynomial perturbation of a polynomial vector 
field with invariant plane supporting a Hamiltonian dynamics. We present 
an explicit example of such perturbation with infinite number of limit cycles 
which accumulate at some separatrix loop. 



1. The result 

Yu. Il'yshenko jn] and J. Ecalle jE] proved that an individual planar polynomial 
vector field can have only finite number of limit cycles. 

On the other hand multi-dimensional vector fields with chaotic dynamics have 



infinite number of periodic trajectories. The Lorentz system jMiMrj and the Duffing 
system |GuHo| provide best known examples. In the chaotic systems the periodic 
orbits are usually encoded by periodic sequences in a suitable symboHc dynamical 
system. This encoding is proved using topological methods (Hke the Lefschetz- 
Coneley index or Smale's horseshoe). This means that: 

(1) The periods of the periodic trajectories tend to infinity in rather irregular 
way. 

(2) The 1-cycles represented by different periodic trajectories have different 
"topology" i.e. they are linked between themselves. 

In particular, these cycles do not form a continuous family (so called center). 

In Main Theorem below we give an example of polynomial 4-dimensional differ- 
ential system, with infinite number of periodic solutions 71,72, .. . such that 

• the periods of 7^ grow monotonically with j; 

• the corresponding 1-cycles have the same "topology"; they are concentric 
cycles on an embedded invariant 2-dimensional disc of class ; 

• the -fj are isolated (they are limit cycles). 

To construct the example we begin with the Hamiltonian planar system 

(1.1) x = Xh = {H,„-H,J, {xuX2)eM.l, H^xl-3xi-xl + 2 
and the 2-dimensional linear system 

(1.2) y^ay, y = yi+ty2eC = Rl, 

where a = — p + icu. Later we put p = oj = \/3. 

The Hamiltonian function from Ijl.lll is elliptic with the critical points x = (—1,0) 
(center) and x = (1,0) (saddle). The phase portrait of the field Xh is shown on 
Figure 11.11 
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(1.3) 



We consider the following coupling of the system Hl.l|l and H1.2|l 

{X = Xh + Rc(Ky) 62 
y = ay + eH\x) {l-xi), 
where e > is a small parameter, 62 = (0, 1) is a versor in Mi^ and k e C. 
Theorem 1.1 (Main). Let p = uj = V3 and 

(1.4) ^ = 4V3* + ^^^1^S^(1 + 2*~V'(¥))' 

where ipiz), the Euler Psi-function, is the logarithmic derivative of the Euler Gamma- 
function V' = ^ . 

Then there exists an eo > such that for any < £ < eo the system kl.!'!\) has a 
sequence of limit cycles 7„, n = 1, 2, . . . which accumulate at the separatrix loop 

7o = {(x,2/): y^O, Hix) = 0, xi < 1} 

of the singular point {x ~ (1, 0), y = 0) and lie on an invariant surface y = eG{x, e) 
of class . 

Remark 1.2. The approximated numerical value of k in formula ljl.4ll is 

K w -0.56 + 4.57i. 

Systems of the form 

j x = Xh + F{x)y + eG{x) + ... 
\y = A{x)y + eb{x) + . . . , 

X e M^, y , i.e. like H1.3|l . appear in the so-called multidimensional general- 
ization of the weakened 16-th Hilbert problem (see |Bol iBZTl lBZ2l lLZ] l. Before 
perturbation, i.e. for £ = 0, we have the invariant plane j/ = with the Hamiltonian 
vector field Xh[x). The ovals H{x) = h form a 1-parameter family of its periodic 
trajectories. One asks how many of these trajectories survive the perturbation. In 
the 2-dimensional case {v = 0) the linearization of the problem leads to the problem 
of real zeroes of an Abelian integral I{h) = JH(x)=h^' called the weakened 
16-th Hilbert problem (see [SHIlI]). 

H v > 1, then the corresponding Pontryagin-Melnikov integrals (see [M] |Pj), 
denoted J{h), were found in |LZj and |BZ1| . We call them the generalized Abelian 
integrals. 



(1.5) 



THE COUNTEREXAMPLE TO A MULTIDIMENSIONAL HILBERT'S. 



3 



The Abelian integrals I{h) satisfy ODEs of the Fuchs type and have regular 
singularities with real spectrum (see |Ya,k| ). Due to this, S. Yakovenko and others 
have found some effective estimations for the number of zeroes of I{h). However, 
the generaHzed Abelian integrals do not satisfy any simple differential equation (see 
|Bo| l and sometimes have irregular singularities (e.g. at h = oo). Moreover, even 
if the singularities are regular, then their spectra can be non-real. 

Namely, the non-reality of the spectrum of J{h) at the singularity /i = is 
responsible for accumulation of zeroes of J. Below we find the asymptotics 

J{h) - C sin(log Vh), h^0+. 

It turns out that the zeroes /i„ — > 0+ of J correspond to Hmit cycles 7„ of the 
system H1.3|l : the cycle 7„ bifurcates form the oval H^^{hn) (see Figure ITT)! . 

Therefore the system can be treated as a counterexample to the multi- 

dimensional weakened Hilbert's problem. 

The remaining parts of the paper are devoted to the proof of Main Theorem. 
In Section 12.11 we investigate the generalized AbeHan integral and its zeroes. In 
Section IT2l we perform estimates needed for existence of genuine limit cycles. 



2. Proof of the Main Theorem 

2.1. Generalized Abelian Integral. The generalized Abelian integral is defined 
in two steps. Firstly one solves the so-called normal variation equation 

(2.1) XHig)=ag+il-xi). 

Its solution X 1-^ g{x) S C appears in the first (linear in e) approximation of the 
invariant surface (see the next section for more details) 

V^eH\x)g{x)+0{e^). 

We consider H2.1|l only in the basin D = {x : H{x) > 0, xi < 1} of the center 
X = (—1,0), filled by the periodic solutions ^h(i) ~ C {H~^{h)}, each of 
period 

(2.2) T,{h) ^ [ -pl^ [ dt. 



2x 



2 



7h 



We assume that the Hamiltonian time is chosen in such a way that for < /i < 4 
x{0) = {x'"i\o), where x'"i\x'"i\x^^^ are roots of the equation H{xi,0) — h ~ 
(see Figure I23J- When restricted to 7/j, the equation Ij2.1|l is treated as the ODE 
g = ag + {1 — xi) with periodic boundary condition. Its unique solution is given in 
the integral form 

(2.3) g{t,h) = ie--^-' -1)-' e'^(*-^)(l - xi)(s, /i) ds. 



Substituting the invariant surface equation (see Section [2]2j y ~ sH^g + 0(£^) 
into the right hand side of x from H1.3II we get the following perturbation of planar 
Hamiltonian system 



(2.4) 



f ii = -2x2, 

\x2 =?,[l-xl) + eH^Rc{ng) + 0[e^). 
The generating function for limit cycles is given by the integral 

(2.5) J{h) ^h'^ 1 Kc{Kg{x)) dxi. 
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Figure 2.1. The basin D filled with ovals 'jh C H ^(/i). 



Let us denote the "basic" generalized Abelian integral (see |BZ1| 1 by 



(2.6) «'^(/i) = / g{t){l - xi){t)dt 



(e-^^^-l)-^ dt dse''(*-^)(l-a;i)(s)(l-2;i)(t). 
'o Jt 



It is related to the generating function via the following 
Lemma 2.1. We have 



J{h) = Rc K (^a'i'^ + 2 j (l-xi) dt^ 



Proof. In this proof we denote by dot, / = ^/ = Xnif), the differential with 
respect to the Hamiltonian time t. We have 

J{h)^-h^ j Re{Kg)-^^{l~ xi)dt ^ h'^Re(Ti J g{l - xi) dty 
Next, g ^ a g + {1 — xi) gives 

J{h) = /i^Re K (^0*^+ J (l-a;i)^dt^ = h'^Rc K(^a1'^+ J {2~-2xi~^X2) dt 

= h*Re ^(a^j + 2 I {1 - xi) dt 



□ 



Our next aim is to determine the leading terms in the asymptotic expansion as 
/i ^ 0+ of the integrals /^(l — xi){t)dt, Tj and We begin with the Abelian 
integrals. It is known |Z2| that these integrals extends to multivalued holomorphic 
functions with logarithmic singularities. We shall need explicit form of the leading 
terms. 
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Lemma 2.2. There exists an open neighborhood G ?7 C C m the complex domain 
and holomorphic functions rjQ , rji , C.o i Ci <= i^{U) such that 

(2.7) ^ rjoih) + Coih) log/i = log/i + ^ log 12 + 0(/ilog/i), 

(2.8) / (1 - xi)dt = Tjiih) + Ci{h) \ogh^2V3 + 0{h\ogh). 



Proof. We consider the pair of basis elliptic Abelian integrals 

Io{h)=T^= / h[h) 



2X2 J-y 2X2 

Note that /^(l - xi)dt = Iq - h and that lo = (area of {H > h}^ . 
These functions (/q, A) satisfy the Picard-Fuchs equations 

6h{h - 4)/^ ^ -{h- 2)Io - 2/i 

6h{h ~ 4)/( = 2/o + (/i - 2)/i . 
The other, independent solution to this system is the pair {Kq,Ki), where 

Js,, 2X2 Js^ 2X2 

are integrals along another cycle 6h in the complex curve Eh = {H{x) = h} C C^. If 
h e (0, 4) then the polynomial xf — 3xi + 2 — /i has three real roots x^^^' < x'"^^ < x^^^ 
(see Figure [OJ- The the cycle 7^1 (respectively Sh) is represented as the lift to the 

Riemann surface Eh of loops in the complex xi-plane surrounding the roots xj^^^* 

(2) (2) (3) 

and x)j (respectively x\ and x)^'). Note the following integral formulas for T^{h): 



f^^^' dxi dxi 

(2.10)7:,= / ^ = = / ^ =, /ie(0,4). 

The second equality corresponds to unobstructed deformation of integration contour 
7h to loop surrounding x^^^ and 00. 

The system (I2.9|l has resonant singular point h ~ Q. Any its solution is either 
analytic near h — Q iii is (i^i, if2)) or it has the form like 

/o(/i) = (ao + ai/i +...)+ 2^ A'o log /i, 

h{h) = {h^ + h^h + ...) + ^^K^ \ogh. 
This representation follows from the Picard-Lefschetz formula 
(2.12) — > ^h ■ Sh, Sh — > Sh, 

which describes the monodromy transformations of the generators of Tri{Eh, *), as 
h surrounds the critical value 0; here * denotes a basepoint. 

We need to calculate the expansions of /o,/i- As we shall see, it is enough to 
calculate i^o(O) and oq; all other coefficients follows from the system Ij2.9|l and can 
be recursively determined. Indeed, to compensate terms with log h in Ij2.ijll we must 
have 

(2.13) 

Terms with h'^ give 
(2.14) 

It can be continued further 



iCi(0) = A-o(0). 
bo^ao + ^ifo(O). 
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To determine Ko{0) and oo simultaneously, we make a coordinate change u = 
{xi — l)/{x^'^^ — 1) in the integral l|2.1fl|l : we denote also p = {x^'^^ — 1). Since 
p — \fhji + 0{h) as ft- ^ 0+, the following integral 



dw 



1 



1 



1 



\Ju^{i + pu) — hjpp- ^v?{i v3i 



1 



0. 



We calculate 

Au 

dw 



1 1 

V3m2 - 3 71 
1 



log 2 
^/3 ' 



+ 



2V3 



VP 



2V3 ' 



log(12V3) 
V3 



Thus flo = log 12, iiro(O) — ~|^- Substituting these values to the relations 
112. Ij2.14|l and using the expansion II2.11|I we get the leading terms of the ex- 
pansions as in formulas (I2.7|l and Ij2.8|l . 

□ 

Let us pass to expansion of 

Proposition 2.3. Let — 2\/3 < Rc(a) < 0. There exists an open neighborhood 
^ U C C in the complex domain and holomorphic functions Lpi,ip2, V3 such that 

(2.15) 

*^(z) = ipi{h) + ^2[h) log h + ipsih) ■ (e-'^^- - 1) = Co + Cift-'^/2x/3 
where 



(2.16) 



(2.17) 



sin2(7ra/2V3)' 



Remark 2.4. One can easily observe that the value H1.4|l of k satisfies the relation 
(2.18) K^i{4:V3 + aCo). 

It is chosen in a way to annihilate the leading term (~ h^) of J{h) and to reveal 



the term with the infinite sequence of zeroes - see the Corollary 12.51 and its proof 
below. 

Corollary 2.5. Providing the values of parameters {p,u!,k) as in Theorem Main, 
the integral J{h) l^2.5\} has a sequence ft„, n — 1,2, . . . of simple zeroes accumulating 
ath = 0. 

Proof. We calculate the leading term of the expansion of J{h) using Lemma 12.11 
Lemma l2?2l and Remark 12.41 

J{h) = h'^Re \k {aCa + aCih^/^~'/^ + 4^/3 + o{h^/'^)) 



h^Re 



h'^Rc 



— n ^l/2-i/2l I /^4+3/4\ 



K (aCo + 4\/3) 
= i?ft4+i/2 cos(log Vft - ao) + o(ft4+3/4), 
where R = \KaCi\, ao = arg(KaCi). Analogously we get 

J'{h) = Ri ft3+i/2 cos(log Vh-ai) + o(/i3+3/4). 
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Thus, by Implicit Function Theorem the zeroes (/i„) of J{h) approximate the simple 
zeroes hn^ of the function cos(log -v/Zi — ao). 

□ 

The remaining part of this section is devoted to the proof of Proposition [231 It 
goes in two steps. In first one we show that the function 

^^(h) Co 

as /i ^ 0"*" and so is bounded. 

In the second step we determine the monodromy of the generalized Abelian 
integral '^-yih) as h surrounds h = 0. We know that then 7 changes to A^ono7 = j-S 
and A1ono(5 = S. We would like to express ^'^.5 in simple terms, in order to 
determine the singularity of '^■y{h) aX h = 0. Rather complicated formulas for ^^.s 
are given m |BZ1] and |BZ2] . In |Bo| these formulas were simplified using certain 
upper triangle representation p of the fundamental group 'Ki{Eh, *). We recall this 
construction below. 

We denote 

^^f^{h) = (e-"^T - J^' dt y*^^' ds e'^(*-")(l - (1 - xi){t), 

cl)^{h) ^X^ / " di / ds (1 - xi){t) (1 - xi){s) ■ e"^^-"'^ = 
Jo Jo 

= A^ / dt d.s(l-a-i)(t+^)(l-xi)(s + ^)e"(*-^\ 



(2.19) 



J-T-,/2 J-T-,/2 

e+ / " dt (1 - xi){t) e"* = / " dt (1 - xi){t + ^) e"*, 

Jo J-T-,/2 

0-=X-' I \\t {I - xi){t) e-"' ^ [ dt{l-xi){t+^)e-''\ 

Jo J-T-,/2 

Here the subscript 7 underlines dependence of the above functions on the loop 
7 = Ih- 

We introduce the following space of triangular matrices 
(2.20) T: ={(0^^^/+). ^eC*, 0+,0-,0eC 

it forms a group. For W we denote 

IM^I = detM^ = A. 

Existence of a 2-dimensional Jordan cell is measured in the following formula 

{W -\ W\){W -1/\W\) 

explicitly we have 



(2.22) = + 

Theorem 2.6 (|Bo]). The map p : tti (£;,,,*) T, 



(2.23) p(7) 



A-y 


^7 
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where X^,9^,cf>^ are defined in k2.19^ . defines a representation of the fundamental 
group of Eh ■ Moreover, we have 

(2.24) *^ = Vop(/i). 

Sketch of the proof. We have = (A^ e"(*-")(l - a:)(t) (l-2;)(s), where 
the integration domain \s ^ {{t, s) : < t < T^, t < s < + t} (see ^J^). 
We divide S into two "triangles" Ai {0 < t < T^, t < s < T^} and A2 = {0 < 
t <T^, < s <T^ + t} = Ao + (0,T^), where Aq = {0 < t < T^, < s < t}. 
We have JJ^^+^^i-) = //ai+Ao(-) + //a.-Ao(-)> where //a,+a„(-) = ^1^7 ^nd 
//a2-Ao(') === (-^7 ~ 1)VV7- Now the formula ^^2^ follows from ES^J. 

The property ^(7 • (5) = ^(7) p{S), 7,(5 G 7ri(iJft, *) is proved analogously. We 
divide the line integrals in and the surface integral in (^.^5 into parts where t or 
s lies in 7 or in 6. We use also = A-yA^. 

□ 

Proposition 2.7. Let ^(t) = (1 — a;i)(i) wii/i the initial value ^(0) = 1 — x^^'' 
('see Fiaure lKT]} . We have the following integral formula for the generalized Abelian 
integral 
(2.25) 

^-,{h) = {e-^^^ - l)-i( / ' eWe'^'dt)' + / ' dt /* ds e(5)^(i)e°(*-^^ 
j4s ft- ^ 0+ i/iese integrals have finite limits: 

^(i)e"*dt ^ ^ ^ = 



(2.26) , 

r dt j ds ^(s)^(t)e'^(*^^) ^ Co, 
where Co,Ci are as defined in Pronosition \2. S[ 

Proof. The value of generalized Abelian integral 5"-^ does not depend on the "shift" 

of parametrization (e.g. t t -\- but values of integrals (p^ and 9^ depend. 

We choose Hamiltonian time parameter in such a way that xi{0) — x\', where 

^i^'' ^ ^i'^'' ^ ^^^^ roots of the polynomial xf — Sxi + 2 = h (see Figure 

ITT|l . Thus 

{l~xi){t + T^/2)^at) 
and so, using formula II2.22|I and formulas H2.19|l . we get formula II2.25|I . 

To determine the asymptotic expansion we notice that the singular curve Eq = 
{H{x) = 0} = {x2 = {xi — l)'^{xi + 2)} is rational. The Hamiltonian parametriza- 
tion of the limit loop 70 can be explicitly calculated: 

(2.27) m^Ut) ^ ^^^J^^^y -oo<t<oo 

as ft ^ 0+. Recall that T'-y(O) = 00. Substituting these values to integrals in Ij2.25(l 
we get the following limits 

^(t)e-*di— . / e'*(°/')dt^V2^.F(go)(aA) 
-T^/2 J -00 cosh (V3t) 



where J-^ denotes the Fourier transform. Since J-{ )(fc) = V2 sinh(fc7r/2) (^^^ 
|(tR| . Integral 3.982.1 for example) we find the value 



' Ci. 



sinh(z7ra/2V3) sin(7ra/2\/3) 
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To determine the limit of the second integral 

-Ty/2 J-T^/2 J~oo J-oc 

we substitute u = t— s, use the symmetry Co(— = Co(i) and the Parsival identity; 
then we get 

/oo \ 2 / \ 



dt 

OO 



3V2i f°° P dk 



TT^^^ J-oo sinh^ kk- m{a/2y^) ' 

To evaluate the latter integral, which has the form 

r°° dz 

(2.28) F(w) = / 5 , Rew < 0, 

J_oo sinh^ z z - mw 

we must use the logarithmic derivative of the Euler F function, i.e. the function 

^ = (iogr)'-E;. 

Integrating by parts we obtain 

F(w) — lim / + / (sgnz — cothz)' dz = 

fl-»+oo_/_^ Jjj-i z ~ niw 

2 2 f-^'^ fi^ coth; 

= —2mw + TT w lim 



R-^+oa J ' Jfi-1 {z — iriw)^ 

Next, we integrate the function along the contour consisting of the segment 

[— i?, —R^^] followed by the semicircle i?~^e*'^, ip g [tt, 27r] and segments: [R^^, R], 
[i?,i? + i(7V+i)7r], [R + i{N+^)TT,-R + i{N+^)Tr], [-R, -R + i{N + ^)ttI where 
A'' G N. Using the residue formula and passing to the Hmit R, N oo we deduce 



^ cothz ni ^ / cothz 

— , + / 7 —T^dz + , . .„ = 27ri } RGSi^„ — ■ 

_R— >+oo J_f; jR-i (z — mw)'' [mw)'' -^-^ \(z ~ mw) 



lim 



2z °° 



= — 7 = — V^-w;); 

TT ^ — ' n — w r TT 

n=0 ^ ' 

in the identification of the latter sum we used |GR| . formula 8.363.8. Finally we 
have 

F(w)=-m{\-2w-2w^t\}\-w)^ for Re(w) < 0, 
and so the second of the limits Ij2.2(i|l follows. 

□ 

Now we investigate the monodromy properties of the generalized Abelian integral 
We shall need the following 

Lemma 2.8 ([Bo|). For W,W' (^T we have 

where [W.W] = WW'W^^ (W^')"^ ^^e commutant and 

^{W) = i\W\^ ~ 1) ijiW); 
(for \W\ = 1 we have i^{W) = 9+ 0- in terms of fTgfl)) ). 
Proof. The proof relies on direct calculations. 

□ 
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Corollary 2.9. The function 'i'-f{h) near h = has the following form 

(2.29) ^,{h) ^ V,{h) + ^^,{h)\0gh- (;,2^f|^\)2 ^[7>](fe) ■ X,{h)-V 

where 6 is the second cycle in Tri{Efi, *) (see the proof of Leminn. \2.^) . ^'[^ 5](/i) = 
'0(p([7, (5]). The functions Lpi, "^s and ^[-y^s] are holomorphic near h = 0. 

Remark 2.10. One can prove that the function ^[-y,*-] is constant 

^ly,s]{h) = (2™)2. 

Indeed, since the contour [7, S] is monodromy invariant (see proof of Corollary 
12.91 belowl. the function ^[-y^s] is meromorphic on whole C with possible poles in 
/i = 0,4. We know, by Proposition 12.71 that it is bounded as /i — > 0. Similarly one 
shows that it is bounded as /i ^ 4. These calculations are analogous to proof of 
the first Hmit in H2.26|l . We also check that ^'[7.5] is bounded as /i ^ 00. Thus this 
function has to be constant; its value we calculate by passing to the limit h ^ 
and comparing respective terms in Ij2.25|l and (I2.29|l . 

Proof of Corollaru \2.fA The Picard-Lefschetz formula II2.12|I . Theorem l2.(il and Lem- 
ma |2]HI imply that 

(2.30) Xono^-^ - V(/o(7) ' PiS)) - + ^-5 + _ i^xj - 1) (X^ Xj ~ 1) ^ '^^ 

and A^ono^'5 = '^s- Next the following monodromy relations follows the Picard- 
Lefschetz formula (12.1211 



A^onoA-y = A-, A^, A^onA^ = A^, 
Mono[l,S] = i-fS) ■ 6 ■ (7<5)-i • 5'^ = [^,5]. 

Therefore ^'5, 5* [7.5] and Xg are locally single- valued functions of h. Since they are 
bounded (see Proposition 12. 7|1 . they must be holomorphic. Now 



A^ono 



Mono (21^*5 log/i) = ^-^s \ogh + -^s 

A5 '^[lA \ _ '^[lA 



(Ai-l)2(A2-l)y (Ai-l)2(A2^Ai-l) 



(A2-l)2(A2-l)y (A2-1)(A2-1)(A2A2-1)- 

Therefore the function ipi defined by H2.29|l is single- valued. Since (A^ — 1) and 
(A^ — 1) are separated from zero and the function '^^ is bounded (see Proposition 
I2.7|l . the function ipi is holomorphic. 

□ 



Corollary 12.91 allows to finish the proof of Proposition 12.31 The holomorphic 
function ipi is defined in Corollary and ip2^ </33 can be read from (I2.29|l : 

^2 = ^5^:7*5, 



2iri 

\2 , 



_ _^|*[^ _ _/27raA5\2 
(A2-l)2- Ia^-iJ 
(the latter equality follows from Remark [2 .1011 . 

Since — "ivl 1°§^ + have the leading term of expansion 

(A2 - l)-i = (e-''^-' - l)-i = + ... 

□ 
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2.2. Estimates. In this subsection we show that the zeroes /i„ of the generaUzed 
AbeHan integral (see Corolla,rv l2.5|l generate corresponding limit cycles of the sys- 
tem (|1.3|l . provided e is sufficiently small. 

Recall that the problem of limit cycles of H1.3II is reduced to the problem of limit 
cycles of the following planar system 

(2.31) x = XH{x)+eRe{7iG{x,e)^e2, 

where the function G{x, e) is defined via the invariant surface ~ {y ~ eG{x, e)}, 
which is a graph of G{-,e). 

At the moment we do not even know whether the invariant surface exists. Indeed, 
the normal hyperbolicity conditions are not satisfied: the eigenvalues in the normal 
direction are A34 = — -s/Si iVS, whereas the eigenvalues at the saddle point x = 
(1,0), y in the x-direction are ±2^3 (compare [BZll lBZ2l IHPSI We 
should do two things: 

(1) prove the existence of the invariant surface, 

(2) estimate the discrepancy G{x,e) — g{x), where g{x) is the solution to 
the normal variation equation given in (j2.1|l . 

In both tasks the crucial role is played by the following Lemma. Let us recall the 
notation related to the elliptic Hamiltonian H{x) = x\ — 2>xi — ^2 + 2. The basin 
D C (see Figure 12.111 is filled with closed orbits of the Hamiltonian vector field 
Xh- 

Lemma 2.11. Let U D D x {0} be an open neighborhood inM? x C and be the 
following vector field in U 

x = Xh + Re{Ky)vo + Q{x, y; e) 
_ y = ay + B{x,y;e), 

where k > 0, a = —\/3 + k £ C, G and Q,B are functions of class 
C'^{U) satisfying the following 

\Q\ < Const- e\H{x)f, 

\B\ < Const - e\H{x)\^. 
Then, for sufficiently small e there exists a unique invariant surface of : 

Le = {{x, y)-- xeD, y^e G{x, e)}. 
The function G{-,e) prolongs by zero to a C^ function on a neighborhood of D in 



(2.32) V, 



p2 



Proof. We shall prove that the Poincare return map associated to vector field 
satisfies the normal hyperbolicity condition. 

In a neighborhood of the center critical point {x = (— l,0),y = 0) for unper- 
turbed vector field Vq, the system is normally hyperbolic and so the invariant surface 
exists. In the further proof we shall concentrate on the neighborhood of separatrix 
70 = {{x, y):y = 0, H{x) = 0} (see Figure[22I). 

The non-degenerate critical point po = ((liO)iO), located on this separatrix, is 
preserved after the perturbation. Let us choose a 3-dimensional hypersurface S 
transversal to 70 and close to the singular point po- Let 

Y.^,d = Sr^{{x,y)■. \y\'' <a'{H{x)f, H{x) < d} 
be a sector in S with vertex 

ctq = S„,dn7o = {(a-,0) e E„,d : H{x) = 0}. 
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Lemma 2.12. For sufficiently small a > 0, d2 > di > and e G (— eo,£o), the 
Poincare return map 

(2.33) Ve ■ Sa.di Sa,d2 

is a diffeomorphism onto the image and prolongs to a map of class in point (Jq. 

Now we finish the proof of Lemma 12.111 The unperturbed Poincare map Vo is 
the identity on the invariant segment /q = Sa.di H {y = 0}. Thus Vq is normally 
hyperbolic on Iq, since we have strong contraction in the normal direction. In 
virtue of the Hirsh Pugh Shub Theorem |HPS| . for sufficiently small e there exist 
the unique invariant embedded interval 1^ close to /q; it is of class . Considering 
Hamiltonian as the parameter on Iq, we get 

I, = {y^eF{h,e), he [0,5)}, F e C\[0,5) x {-eo,eo)). 

The surface spanned by trajectories of passing through is V^-invariant, due 
to the invariance of /e under the Poincare map ■ The form of invariant interval 
/e and the form of vector field implies that the surface is graph of a (D) 
function: 

Le = {{x,y) : X e D, y^eG{x,e)}. 

We prove that G can be extended by zero outside D. 

We check that, providing the assumptions of Lemma, 12.111 hold, the set 

{{x,y):xeD,\y\' <R^e^\H{x)\'}, 

for R big enough, is invariant for V^. Namely, denoting all constants by C, we 
calculate 

(2.34) Ve{\y\' - R'e'\H\%y^=i,,,^H\^ = 



= 2Re 



y{ay + B) - 4R^ H^Re{K y <dH,VQ> + < dH,Q >] 



< 2R^e^ H'^i - p+ % + 2ReC\H\^+'' + eC\H\ 



THE COUNTEREXAMPLE TO A MULTIDIMENSIONAL HILBERT'S. 



13 



Since the latter expression is < (for sufficiently large R), the considered subset is 
-invariant. 
Thus 

\G{x,e)\ < Const - \H{x)\^ 
and the function G(a;,e) can be prolonged by zero to a function of class C^. 

□ 

Proof of Lfimmffi fg.jgL By calculations analogous to H2.34|l we find that 

- a'H%y^=^ <2a^H'(^-p+i^ + aC\H\'' + Ce \H\). 

Thus for sufficiently small a and e the subset 

{{x,y):xeD, \y\^ < a\H{x)f} 

is V^-invariant. Moreover, the separatrix 70 is also T4-invariant. This proves, that 
the Poincare return map defines the dyffeomorphism lj2Ji;^|l which is of class 
outside the border. 

Thus it remains to show that Vg can be prolonged to the map in the point 
ctq. We choose an additional, auxiliary, 3-dimensional hypersurface S transversal to 
7o, close to po, which Hes "on another side" with respect to the point po (see Figure 
I2.2|l . The return map Ve is the composition Ve ~ o Vl of the correspondence 
maps 

ri:S^ S, and : S S, 

defined by trajectories near the singular point po and trajectories near the regular 
part of 7o respectively. The regular map naturally extends to the map (even 
in fact) as the fiow of the non- vanishing, vector field . To analyze the singular 
part V^, we use the following theorem of H. Belitskii. 

Theorem 2.13 f |Be| l. Let A e End(]R") be a linear endomorphism whose eigen- 
values (Ai, . . . , A„) satisfy 

RcXi ^ RcXj + ReAfc 
for all i and j, k such that RcAj < < RcAfe . Then any differential system 

— = A.X + f[x), /(O) = = /'(O) 

is in the neighborhood of -equivalent to the linearization. 

The eigenvalues of the Hnearization of our vector field in po are ±2\/3, —^/3± 
i-\/3, so Vg satisfies the assumptions of the Belitskii theorem. In suitable coordi- 
nates (u,w), associated with the Hnearization of in the neighborhood of po, the 
correspondence map has the form 

r^iu,v) ^ {u,C u^v), ueR+, we(C,0), /? = i - f . 

The restriction to Y^a,d corresponds to the restriction to the set < a{u)u, 

u e R_|-, 3(0) > 0}. In such region the map is of class in ctq; we have 
(w, u)((To) = (0, 0) and ("Pf )'(0, 0) = ( ^ •] ). Thus, the thesis of the lemma follows. 

□ 

Now we can prove the existence of the invariant surface and estimate the distance 
to its linear approximation H'^g. 

Proposition 2.14. For sufficiently small e, there exists the invariant surface 
(2.35) Lg = {y = eG{x,e), x C D} 
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of the system fi.,^) . The function G is of class and the distance to the lineariza- 
tion {H^g){x) = G(x, 0) is hounded by 

(2.36) \G-H^g\<C\e\\h\\ 

(2.37) \G'-{H^gy\<C\e\\h\\ 
where G' = G^x is the derivative with respect to x. 

Proof. The existence of the invariant surface of class G^ and the form II2.35|I is a 
direct consequence of Lemma [2 .111 

To show the bounds H2.36|l . (|2.37|l . we make a coordinate change 

(x, y) ^ (x, z), z=(y~e H^g{x)) /(e H^). 

The system lll.3|l takes the form 

f X =Xh + -ff^ Rc(kz) e 62 + £ -ff^ Re(K g) 62, 



(2.38) 



i =a z + 4e + 5)Rc(k (z + g)) 2.T2 -eH^^^ Re(K (z + g)). 



Using the integral formula Ij2.3ll for the function g we deduce that it is bounded, 
g < C. Since the function g prolongs to the (multivalued), holomorphic function 
ramified along the singular curve 70, the following bounds for derivatives of g hold 

(2.39) 19^"^ < Ck\H\-\ 

Using this one can check that the system H2.38|l satisfies the assumptions of Lemma 
12.111 Thus, the invariant surface has the form 

eG-eH*g 



z = 



e U{x, e) 



and the function U prolongs by zero to a G^ function on a neighborhood of D. 
Thus, the function U satisfies the estimates 

\U\<G\Hl \U'\<G, 

which are equivalent to H2.36|l and Ij2.37|l . 

□ 

Now we show that the generahzed Abelian integral J[h) is a good approximation 
of the Poincare return map and so the zeroes of J{h) generate limit cycles for 
sufficiently small e. 

Proposition 2.15. Let AH{h,e) be the increment of the Hamiltonian after the 
first return of the system restricted to the invariant surface Lg. Then, there 
exists a constant G such that 

(2.40) |Ai/-£j| < Ce|/^|^ 

(2.41) \dh{AH)-eJ'{h)\<Ge\h\^ \ \ogh\. 

Proof. Here we study the phase curves of the 2-dimensional vector field II2.31|I i.e. 
We-. =V,\L,^XH+eReiTiG)dx,. 

We fix the segment / ~ {(a;i,0) : xi e [—2, —1)} transversal to the Hamiltonian 
fiow. We denote by /3e(t, /i) the integral curves of which start and finish at /. 
They satisfy 

l3e{t, h) = Xh + eRe(KG) 62, 

(2.42) /3,(0,/i)e/, /3,(Te,Me/ 
H{l3Mh)) = h. 
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For £ = the curve /3o is the oval {H = h} and for e non-zero but small it is a 
small perturbation of Po- 

I3e{t,h) ^ l3o{t,h) + eb{t,h-e). 

Above = T^{h) is the time of the first return to the unit /. 

Lemma 2.16. There exist a constant C and the positive, small constant v such 
that the following estimates hold: 

(2.43) |6| < C|/i|4 6(2^+")*, 

(2.44) \dhb\ < C\h\^ e^2V3+^)\ 

(2.45) \Te-To\<Ce\h\^ 



Proof. We use the scalar product x ■ x' = 3xi x[ + X2 X2 and = ^/x ■ x. 

It follows from the equation Ij2.42ll that the function b satisfies the following 
initial value problem 



(2.46) 



6 = cIXh b + Re(KG'(/3o + e 6)) 62, 
6(0, /i;e) = 0, 



where dXjy6= ^ {{X h {(3o + e b) - X h (Pq)) . We have dX^ ^ dXH{f3o + eb), ioi 
some 9 G (0, 1). Hence 

^H = {^Lro)^ :^-ieh2,l]. 

Moreover, using estimates lf23fi|l . ifOTj) . ifOoll we get |G'(/3o + £6)| < Ci h-^. For 
the solution 6 to the equation H2.4fill we have 



^|6p = 2|6| ^|6| = 26-dXH6 + 26-Rc(KG(/3o+e6))e2 = 

= -12{xi + 1) 61 62 + 262 Rc(kG(/3o + e6)) < 4\/3|6p + 2C2h^\b\. 

Therefore ^ (|b|) < 2 x/3|6| + C2/^^ |6|(0) = and the Gronwall inequality [HI 
gives the bound (I2.43|l . 

Since the difference of fiows eb after the Hamiltonian period Tq is < C\h\^ and 
the "velocity" \Ve \ ~ 1, the difference of periods jT^ - TqI satisfies H2.45II . 

The derivative ^ satisfies the respective linear variation equation related to 

i (f ) = {dXH + e{.. .)) f + /.^ (. . .) (f^), f (0, h; e) = 0, 
where we denoted by (. . .) the bounded terms (see estimations H2. 4312. 4512.3712. 



Since the fiow variation of the Hamiltonian field satisfies < Ce^^*, the 

Oh Oh — ' 

inequality Ij2.44ll holds. This finishes the proof of Lemma [2. 16 

□ 

We continue the proof of Provosition \2. 1 ,51 

We split the difference between the Poincare map and the linearization eJ{h) in 
two integrals Ri{h,e) and R2{h,e): 

Ri= I Rc{k {G - H'^g)) dxi, 

R2^ I Yic{-KH^g)dxi~ I Yic{liH'^g) dxi. 

■lie Jla 

We shall show that the estimations 112.4011 and (12. 4111 hold for both Ri and R2 . 
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The inequality Ij2.40|l is a direct consequence of the bound l|2.36ll . The difference 
of i?i in close values of h takes the form 

Ri{h + S) - Ri{h) = [ d^,^Re{K{G-H^g))dx2/\dxi+0{e\h~2\^), 

where the integral is taken along the strip 7e(/i, h + 5) between 7e(/i) and 7e(/i + (5). 
The area of this strip is of the same order as the area of the domain {X : h < H < 
h + S}, i.e. ^ S ■ lo ^ C S \ \ogh\ (see proof of Lemma, [T2)l . So the estimate H2.41|l 

follows llOTji . 

To prove the estimate for i?2 we use (j2.42ll and Ij2.39|l : 

|i?2| <Ci [ " Rc(Kij4g)(/3o + eb) - Re{K H^g){/3o) + / ' C2 £ \hf < 

Jo Jto 

<C3e\h\'^ dt + C4\h\'^\T,^To\ <C5e|/i|'^-2" <Ce|/i|^ 

Jo 

Similarly, using the following formula for differential of integral 

and the bounds l|2.42ll . (I2.39|l . we get 

\R2\<Ce\hf, \dhR2\<Ce\h\\ 

□ 

Now we can finish the proof of the Main Theorem. The restriction of system (ll.3|l 
to its invariant surface ^ {y = eG{x,e)} has the form H2.31|l . The increment 
/S.H = 'P[h) — h, associated with the Poincare map V{h) (on a section transversal 
to 7o), equals (see Pronosition l2.15ll 

Ai/(/i)=eJ(/i) + 0(|e||/i|^), 

(AHYih) = eJ\h) + o{\e\\h\^-^/^). 

Since we also have (see proof of Corollarv l2.5|l 

J{h) = Rh'^+^/'^ cos(log Vh-ao)+ o{h'^-^/^), 

any simple zero of J{ti), which is sufficiently close to generates, by the Implicit 
Function Theorem, a simple zero of /S.H . Thus the sequence /i„ 0+ of sim- 
ple zeroes of J{h) (see Corolla,rv l2.5|l guarantees the existence of infinite sequence 

^ 0+, n > iVo of simple zeroes of the increment l\H . Any simple zero of AiJ 
corresponds to a limit cycle of H1.3|l . 

The proof of Main Theorem is now complete. 

□ 



References 

[AI] Arnold V. I. and Il'yashenko Yu. S., Ordinary differential equations, in: "Ordinary Dif- 
ferential Equations and Smooth Dynamical Systems", Springer- Verlag, New York, (1997), 
pp. 1-148; (Russian: Fundamental Directions, v. 1, VINITI, Moscow, 1985, pp. 1-146). 

[Be] Belitskii H., "Normal forms, invariant and local mappings", Naukova Dumka, Kiev, 1979 
(in Russian). 

[Bo] Bobienski M., Contour integrals as Ad-invariant functions on the fundamental group (sub- 
mitted). 

[BZl] Bobienski M. and Zol^dek H., Limit cycles for multidimensional vector field. The elliptic 

case, J. Dynam. Control Systems, 9, (2003), No 2, 265-310. 
[BZ2] Bobienski M. and Zolq,dek H., Limit cycles of three dimensional polynomial vector fields, 

Nonlinearity 18, (2005), No 1, 175-209. 



THE COUNTEREXAMPLE TO A MULTIDIMENSIONAL HILBERT'S. 



17 



[E] Ecalle J., "Introduction aux fonctions analysables et preuve constructivede la conjecture 
de Dulac", Actualites Mathematiques, Herman, Paris, 1992. 

[GR] Gradshteyn I. S., Ryzhik I. M., "Table of Integrals, Series and Products. Fifth edition". 
Academic Press, Inc. 1994. 

[GuHo] Guckenheimer J. and Holmes P., "Nonlinear oscillations, dynamical systems, and bifurca- 
tions of vector fields". Applied Mathematical Sciences, 42, Springer- Verlag, 1983. 

[H] Hartman, P., "Ordinary differential equations". Classics in Applied Mathematics, 38, 
Philadelphia, PA, 2002. 

[HPS] Hirsch M., Pugh C., Shub M., "Invariant manifolds", Lect. Notes in Math. 583, Springer- 
Verlag, New York, 1977. 

[II] Il'yashenko Yu. S., Centennial history of Hilbert's 16th problem. Bull. Amer. Math. Soc. 

39 (2002), No 3, 301-354. 
[M[ Melnikov V. K., On the stability of a center for time-periodic perturbations, Trans. Moscow 

Math. Soc. 12 (1963), 1-57. 
[MiMr[ Mischaikow K., Mrozek M., Chaos in the Lorenz equations: a computer-assisted proof, 

Bull. Amer. Math. Soc. (N.S.), 32, (1995), No 1, 66-72. 
[Ni[ Nitecki Z., "Differentiable dynamics. An introduction to the orbit structure of diffeomor- 

phisms", MIT Press, Cambridge, 1971. 
[P[ Pontryagin L. S., On dynamical systems close to Hamiltonian systems, in: "Selected 

Works", V. 1, Gordon & Breach, New York, 1986; [Russian: Zh. Ekper. Teoret. Fiziki 

4 (19,34), 234-238[. 

[LZ] Leszczynski P., Zol^dek H., Limit cycles appearing after perturbation of certain multi- 
dimensional vector fields, J. Dynam. Diff. Equat. 13 (2001), No 4, 689-709. 

[Yak] S. Yakovenko, On functions and curves defined by ordinary differential equations, The 
Arnoldfest (Toronto, ON, 1997), Fields Inst. Commun., 24, Amer. Math. Soc, Providence, 
1999, pp. 497-525. 

]Z2] H. Zol^dek, "The Monodromy Group", Monografie Matematyczne, Birkhauser, Basel, 2006. 

Institute of Mathematics, Warsaw University, ul. Banacha 2, 02-097 Warsaw, 
Poland 

E-mail address: mbobi8mimuu.edu.pl 
E-mail address: zoladekSmimuw. edu.pl 



